Multi-Class Model for Human Activity Recognition Using Scikit-Learn Take 1

David Lowe

June 10, 2020

Template Credit: Adapted from a template made available by Dr. Jason Brownlee of Machine Learning Mastery. [https://machinelearningmastery.com/]

SUMMARY: The purpose of this project is to construct a predictive model using various machine learning algorithms and to document the end-to-end steps using a template. The Human Activity Recognition Using Smartphones dataset is a multi-class classification situation where we are trying to predict one of several (more than two) possible outcomes.

INTRODUCTION: Researchers collected the datasets from experiments that consist of a group of 30 volunteers, with each person performing six activities by wearing a smartphone on the waist. With its embedded accelerometer and gyroscope, the research captured measurement for the activities of WALKING, WALKING_UPSTAIRS, WALKING_DOWNSTAIRS, SITTING, STANDING, LAYING. The dataset has been randomly partitioned into two sets, where 70% of the volunteers were selected for generating the training data and 30% the test data.

In previous iterations, the script focused on evaluating various classic machine learning algorithms and identify the algorithm that produces the best accuracy metric. The previous iterations established a baseline performance in terms of accuracy and processing time.

In this Take1 iteration, we will construct and tune an XGBoost machine learning model for this dataset. We will observe the best accuracy result that we can obtain using the XGBoost model with the training and test datasets.

ANALYSIS: For this Take1 iteration, the XGBoost model achieved an accuracy metric of 99.45% in training. When configured with the optimized parameters, the XGBoost model processed the test dataset with an accuracy of 94.94%, which indicated a high variance issue. We will need to explore regularization techniques or other modeling approaches before deploying the model for production use.

CONCLUSION: For this iteration, the XGBoost algorithm achieved the best overall results using the training and test datasets. For this dataset, Random Forest should be considered for further modeling.

Dataset Used: Human Activity Recognition Using Smartphones

Dataset ML Model: Multi-class classification with numerical attributes

Dataset Reference: https://archive.ics.uci.edu/ml/datasets/Human+Activity+Recognition+Using+Smartphones

Any predictive modeling machine learning project generally can be broken down into about six major tasks:

1. Prepare Environment
2. Summarize Data
3. Prepare Data
4. Model and Evaluate Algorithms
5. Improve Accuracy or Results
6. Finalize Model and Present Results

Task 1. Prepare Environment

!pip install python-dotenv PyMySQL

Collecting python-dotenv
  Downloading https://files.pythonhosted.org/packages/cb/2a/07f87440444fdf2c5870a710b6770d766a1c7df9c827b0c90e807f1fb4c5/python_dotenv-0.13.0-py2.py3-none-any.whl
Collecting PyMySQL
  Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB)
     |████████████████████████████████| 51kB 2.4MB/s 
Installing collected packages: python-dotenv, PyMySQL
Successfully installed PyMySQL-0.9.3 python-dotenv-0.13.0

# Retrieve CPU information from the system
ncpu = !nproc
print("The number of available CPUs is:", ncpu[0])

The number of available CPUs is: 2

# Retrieve GPU configuration information from Colab
gpu_info = !nvidia-smi
gpu_info = '\n'.join(gpu_info)
if gpu_info.find('failed') >= 0:
    print('Select the Runtime → "Change runtime type" menu to enable a GPU accelerator, ')
    print('and then re-execute this cell.')
else:
    print(gpu_info)

Sun Jun  7 20:52:10 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.82       Driver Version: 418.67       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |
| N/A   35C    P0    25W / 250W |      0MiB / 16280MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

# Retrieve memory configuration information from Colab
from psutil import virtual_memory
ram_gb = virtual_memory().total / 1e9
print('Your runtime has {:.1f} gigabytes of available RAM\n'.format(ram_gb))

if ram_gb < 20:
    print('To enable a high-RAM runtime, select the Runtime → "Change runtime type"')
    print('menu, and then select High-RAM in the Runtime shape dropdown. Then, ')
    print('re-execute this cell.')
else:
    print('You are using a high-RAM runtime!')

Your runtime has 13.7 gigabytes of available RAM

To enable a high-RAM runtime, select the Runtime → "Change runtime type"
menu, and then select High-RAM in the Runtime shape dropdown. Then, 
re-execute this cell.

1.a) Load libraries and modules

# Set the random seed number for reproducible results
seedNum = 888

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import os
import sys
import smtplib
from datetime import datetime
from email.message import EmailMessage
from dotenv import load_dotenv
from sklearn import preprocessing
from sklearn.model_selection import train_test_split
from sklearn.model_selection import KFold
from sklearn.model_selection import StratifiedKFold
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
from sklearn.metrics import roc_auc_score
from xgboost import XGBClassifier

1.b) Set up the controlling parameters and functions

# Begin the timer for the script processing
startTimeScript = datetime.now()

# Set up the number of CPU cores available for multi-thread processing
# n_jobs = int(ncpu[0])

# Set up the flag to stop sending progress emails (setting to True will send status emails!)
notifyStatus = False

# Set up the parent directory location for loading the dotenv files
useColab = False
if useColab:
    # Mount Google Drive locally for storing files
    from google.colab import drive
    drive.mount('/content/gdrive')
    gdrivePrefix = '/content/gdrive/My Drive/Colab_Downloads/'
    env_path = '/content/gdrive/My Drive/Colab Notebooks/'
    dotenv_path = env_path + "python_script.env"
    load_dotenv(dotenv_path=dotenv_path)

# Set up the dotenv file for retrieving environment variables
useLocalPC = False
if useLocalPC:
    env_path = "/Users/david/PycharmProjects/"
    dotenv_path = env_path + "python_script.env"
    load_dotenv(dotenv_path=dotenv_path)

# Configure the plotting style
plt.style.use('seaborn')

# Set Pandas options
pd.set_option("display.max_rows", 120)
pd.set_option("display.width", 140)

# Set the flag for splitting the dataset
splitDataset = False
# splitPercentage = 0.25

# Set the number of folds for cross validation
n_folds = 5

# Set various default modeling parameters
scoring = 'accuracy'

1.c) Load dataset

dataset_url = "https://dainesanalytics.com/datasets/ucirvine-human-activity-recognition/"
widthVector = [16] * 561
colNames = ["attr" + str(i).zfill(3) for i in range(1,562)]
X_train_df = pd.read_fwf(dataset_url+'train/X_train.txt', widths=widthVector, header=None, names=colNames)
y_train_df = pd.read_csv(dataset_url+'train/y_train.txt', names=["targetVar"])
Xy_train_df = pd.concat([X_train_df, y_train_df], axis=1)
X_test_df = pd.read_fwf(dataset_url+'test/X_test.txt', widths=widthVector, header=None, names=colNames)
y_test_df = pd.read_csv(dataset_url+'test/y_test.txt', names=["targetVar"])
Xy_test_df = pd.concat([X_test_df, y_test_df], axis=1)
Xy_original = pd.concat([Xy_train_df, Xy_test_df], axis=0)

# Take a peek at the dataframe after import
Xy_original.head(10)

	attr001	attr002	attr003	attr004	attr005	attr006	attr007	attr008	attr009	attr010	attr011	attr012	attr013	attr014	attr015	attr016	attr017	attr018	attr019	attr020	attr021	attr022	attr023	attr024	attr025	attr026	attr027	attr028	attr029	attr030	attr031	attr032	attr033	attr034	attr035	attr036	attr037	attr038	attr039	attr040	...	attr523	attr524	attr525	attr526	attr527	attr528	attr529	attr530	attr531	attr532	attr533	attr534	attr535	attr536	attr537	attr538	attr539	attr540	attr541	attr542	attr543	attr544	attr545	attr546	attr547	attr548	attr549	attr550	attr551	attr552	attr553	attr554	attr555	attr556	attr557	attr558	attr559	attr560	attr561	targetVar
0	0.288585	-0.020294	-0.132905	-0.995279	-0.983111	-0.913526	-0.995112	-0.983185	-0.923527	-0.934724	-0.567378	-0.744413	0.852947	0.685845	0.814263	-0.965523	-0.999945	-0.999863	-0.994612	-0.994231	-0.987614	-0.943220	-0.407747	-0.679338	-0.602122	0.929294	-0.853011	0.359910	-0.058526	0.256892	-0.224848	0.264106	-0.095246	0.278851	-0.465085	0.491936	-0.190884	0.376314	0.435129	0.660790	...	-0.991364	-1.0	-0.936508	0.346989	-0.516080	-0.802760	-0.980135	-0.961309	-0.973653	-0.952264	-0.989498	-0.980135	-0.999240	-0.992656	-0.701291	-1.000000	-0.128989	0.586156	0.374605	-0.991990	-0.990697	-0.989941	-0.992448	-0.991048	-0.991990	-0.999937	-0.990458	-0.871306	-1.000000	-0.074323	-0.298676	-0.710304	-0.112754	0.030400	-0.464761	-0.018446	-0.841247	0.179941	-0.058627	5
1	0.278419	-0.016411	-0.123520	-0.998245	-0.975300	-0.960322	-0.998807	-0.974914	-0.957686	-0.943068	-0.557851	-0.818409	0.849308	0.685845	0.822637	-0.981930	-0.999991	-0.999788	-0.998405	-0.999150	-0.977866	-0.948225	-0.714892	-0.500930	-0.570979	0.611627	-0.329549	0.284213	0.284595	0.115705	-0.090963	0.294310	-0.281211	0.085988	-0.022153	-0.016657	-0.220643	-0.013429	-0.072692	0.579382	...	-0.991134	-1.0	-0.841270	0.532061	-0.624871	-0.900160	-0.988296	-0.983322	-0.982659	-0.986321	-0.991829	-0.988296	-0.999811	-0.993979	-0.720683	-0.948718	-0.271958	-0.336310	-0.720015	-0.995854	-0.996399	-0.995442	-0.996866	-0.994440	-0.995854	-0.999981	-0.994544	-1.000000	-1.000000	0.158075	-0.595051	-0.861499	0.053477	-0.007435	-0.732626	0.703511	-0.844788	0.180289	-0.054317	5
2	0.279653	-0.019467	-0.113462	-0.995380	-0.967187	-0.978944	-0.996520	-0.963668	-0.977469	-0.938692	-0.557851	-0.818409	0.843609	0.682401	0.839344	-0.983478	-0.999969	-0.999660	-0.999470	-0.997130	-0.964810	-0.974675	-0.592235	-0.485821	-0.570979	0.273025	-0.086309	0.337202	-0.164739	0.017150	-0.074507	0.342256	-0.332564	0.239281	-0.136204	0.173863	-0.299493	-0.124698	-0.181105	0.608900	...	-0.986658	-1.0	-0.904762	0.660795	-0.724697	-0.928539	-0.989255	-0.986028	-0.984274	-0.990979	-0.995703	-0.989255	-0.999854	-0.993238	-0.736521	-0.794872	-0.212728	-0.535352	-0.871914	-0.995031	-0.995127	-0.994640	-0.996060	-0.995866	-0.995031	-0.999973	-0.993755	-1.000000	-0.555556	0.414503	-0.390748	-0.760104	-0.118559	0.177899	0.100699	0.808529	-0.848933	0.180637	-0.049118	5
3	0.279174	-0.026201	-0.123283	-0.996091	-0.983403	-0.990675	-0.997099	-0.982750	-0.989302	-0.938692	-0.576159	-0.829711	0.843609	0.682401	0.837869	-0.986093	-0.999976	-0.999736	-0.999504	-0.997180	-0.983799	-0.986007	-0.627446	-0.850930	-0.911872	0.061436	0.074840	0.198204	-0.264307	0.072545	-0.155320	0.323154	-0.170813	0.294938	-0.306081	0.482148	-0.470129	-0.305693	-0.362654	0.507459	...	-0.988055	-1.0	1.000000	0.678921	-0.701131	-0.909639	-0.989413	-0.987836	-0.986850	-0.986749	-0.996199	-0.989413	-0.999876	-0.989136	-0.720891	-1.000000	-0.035684	-0.230091	-0.511217	-0.995221	-0.995237	-0.995722	-0.995273	-0.995732	-0.995221	-0.999974	-0.995226	-0.955696	-0.936508	0.404573	-0.117290	-0.482845	-0.036788	-0.012892	0.640011	-0.485366	-0.848649	0.181935	-0.047663	5
4	0.276629	-0.016570	-0.115362	-0.998139	-0.980817	-0.990482	-0.998321	-0.979672	-0.990441	-0.942469	-0.569174	-0.824705	0.849095	0.683250	0.837869	-0.992653	-0.999991	-0.999856	-0.999757	-0.998004	-0.981232	-0.991325	-0.786553	-0.559477	-0.761434	0.313276	-0.131208	0.191161	0.086904	0.257615	-0.272505	0.434728	-0.315375	0.439744	-0.269069	0.179414	-0.088952	-0.155804	-0.189763	0.599213	...	-0.994169	-1.0	-1.000000	0.559058	-0.528901	-0.858933	-0.991433	-0.989059	-0.987744	-0.991462	-0.998353	-0.991433	-0.999902	-0.989321	-0.763372	-0.897436	-0.273582	-0.510282	-0.830702	-0.995093	-0.995465	-0.995279	-0.995609	-0.997418	-0.995093	-0.999974	-0.995487	-1.000000	-0.936508	0.087753	-0.351471	-0.699205	0.123320	0.122542	0.693578	-0.615971	-0.847865	0.185151	-0.043892	5
5	0.277199	-0.010098	-0.105137	-0.997335	-0.990487	-0.995420	-0.997627	-0.990218	-0.995549	-0.942469	-0.565684	-0.822766	0.849095	0.695586	0.845922	-0.993928	-0.999985	-0.999857	-0.999917	-0.997584	-0.991847	-0.995414	-0.751869	-0.454773	-0.550882	0.390052	-0.182272	0.158751	0.187313	0.259951	-0.243230	0.421736	-0.418460	0.558461	-0.218344	0.165138	0.080920	-0.209979	-0.151064	0.180424	...	-0.996697	-1.0	-1.000000	0.246910	-0.520879	-0.802525	-0.990500	-0.985861	-0.985512	-0.988204	-0.994758	-0.990500	-0.999861	-0.991967	-0.768577	-1.000000	-0.297329	-0.346045	-0.727270	-0.995143	-0.995239	-0.994398	-0.996088	-0.998519	-0.995143	-0.999974	-0.994530	-1.000000	-1.000000	0.019953	-0.545410	-0.844619	0.082632	-0.143439	0.275041	-0.368224	-0.849632	0.184823	-0.042126	5
6	0.279454	-0.019641	-0.110022	-0.996921	-0.967186	-0.983118	-0.997003	-0.966097	-0.983116	-0.940987	-0.565684	-0.817189	0.851040	0.674347	0.833591	-0.986837	-0.999982	-0.999658	-0.999634	-0.996863	-0.972402	-0.982864	-0.637185	-0.514751	-0.536553	0.360460	-0.233335	0.226457	0.069525	0.064297	-0.076439	0.138057	-0.036769	0.231401	-0.114577	0.319204	-0.487769	-0.095852	-0.135729	0.610845	...	-0.991345	-1.0	-0.904762	0.290177	-0.668835	-0.933443	-0.988269	-0.984569	-0.983601	-0.985328	-0.994521	-0.988269	-0.999828	-0.989040	-0.735639	-1.000000	-0.257032	-0.321591	-0.658435	-0.995641	-0.994639	-0.994202	-0.994491	-0.996934	-0.995641	-0.999974	-0.993940	-0.955696	-1.000000	0.145844	-0.217198	-0.564430	-0.212754	-0.230622	0.014637	-0.189512	-0.852150	0.182170	-0.043010	5
7	0.277432	-0.030488	-0.125360	-0.996559	-0.966728	-0.981585	-0.996485	-0.966313	-0.982982	-0.940987	-0.572638	-0.817189	0.850328	0.670410	0.832383	-0.976851	-0.999980	-0.999341	-0.999164	-0.995768	-0.975847	-0.986829	-0.633239	-0.743877	-0.822408	0.363064	-0.300719	0.346867	-0.063694	0.144124	-0.132349	0.083232	0.041631	0.137192	-0.092290	0.372369	-0.720913	-0.161634	-0.017120	0.561421	...	-0.989676	-1.0	-0.968254	0.249799	-0.655503	-0.932563	-0.989431	-0.987065	-0.985978	-0.988193	-0.998240	-0.989431	-0.999867	-0.990035	-0.735639	-1.000000	-0.197272	-0.407284	-0.733313	-0.995629	-0.994507	-0.994727	-0.994018	-0.997006	-0.995629	-0.999974	-0.995423	-0.955696	-1.000000	0.136382	-0.082307	-0.421715	-0.020888	0.593996	-0.561871	0.467383	-0.851017	0.183779	-0.041976	5
8	0.277293	-0.021751	-0.120751	-0.997328	-0.961245	-0.983672	-0.997596	-0.957236	-0.984379	-0.940598	-0.564175	-0.823527	0.850328	0.670410	0.832383	-0.983295	-0.999985	-0.999528	-0.999426	-0.997351	-0.957212	-0.985841	-0.683117	-0.524812	-0.758524	0.358268	-0.253309	0.266929	-0.124250	0.215324	-0.183362	0.171745	-0.068957	0.241831	-0.194419	0.216758	-0.338026	-0.164791	-0.033767	0.724234	...	-0.992371	-1.0	-1.000000	0.272342	-0.750485	-0.929474	-0.991629	-0.991759	-0.991182	-0.991522	-0.995197	-0.991629	-0.999927	-0.991428	-0.850310	-1.000000	0.073417	-0.371389	-0.674770	-0.995260	-0.996007	-0.995828	-0.995482	-0.993608	-0.995260	-0.999977	-0.995095	-0.955696	-1.000000	0.314038	-0.269401	-0.572995	0.012954	0.080936	-0.234313	0.117797	-0.847971	0.188982	-0.037364	5
9	0.280586	-0.009960	-0.106065	-0.994803	-0.972758	-0.986244	-0.995405	-0.973663	-0.985642	-0.940028	-0.554594	-0.815850	0.845442	0.684757	0.838455	-0.986541	-0.999963	-0.999662	-0.999718	-0.996163	-0.979809	-0.982811	-0.550721	-0.294553	-0.479711	0.192863	-0.021212	0.037915	0.190540	0.039722	0.054283	0.037268	0.081551	0.172340	0.035995	-0.184930	0.351156	-0.176241	-0.447651	0.391867	...	-0.979144	-1.0	-1.000000	0.250884	-0.510116	-0.801919	-0.984788	-0.980388	-0.981850	-0.976771	-0.993197	-0.984788	-0.999736	-0.981969	-0.671366	-1.000000	-0.139966	0.106785	-0.131360	-0.990981	-0.990599	-0.991665	-0.988199	-0.984877	-0.990981	-0.999929	-0.990456	-0.955696	-1.000000	0.267383	0.339526	0.140452	-0.020590	-0.127730	-0.482871	-0.070670	-0.848294	0.190310	-0.034417	5

10 rows × 562 columns

Xy_original.info(verbose=True)

<class 'pandas.core.frame.DataFrame'>
Int64Index: 10299 entries, 0 to 2946
Data columns (total 562 columns):
 #   Column     Dtype  
---  ------     -----  
 0   attr001    float64
 1   attr002    float64
 2   attr003    float64
 3   attr004    float64
 4   attr005    float64
 5   attr006    float64
 6   attr007    float64
 7   attr008    float64
 8   attr009    float64
 9   attr010    float64
 10  attr011    float64
 11  attr012    float64
 12  attr013    float64
 13  attr014    float64
 14  attr015    float64
 15  attr016    float64
 16  attr017    float64
 17  attr018    float64
 18  attr019    float64
 19  attr020    float64
 20  attr021    float64
 21  attr022    float64
 22  attr023    float64
 23  attr024    float64
 24  attr025    float64
 25  attr026    float64
 26  attr027    float64
 27  attr028    float64
 28  attr029    float64
 29  attr030    float64
 30  attr031    float64
 31  attr032    float64
 32  attr033    float64
 33  attr034    float64
 34  attr035    float64
 35  attr036    float64
 36  attr037    float64
 37  attr038    float64
 38  attr039    float64
 39  attr040    float64
 40  attr041    float64
 41  attr042    float64
 42  attr043    float64
 43  attr044    float64
 44  attr045    float64
 45  attr046    float64
 46  attr047    float64
 47  attr048    float64
 48  attr049    float64
 49  attr050    float64
 50  attr051    float64
 51  attr052    float64
 52  attr053    float64
 53  attr054    float64
 54  attr055    float64
 55  attr056    float64
 56  attr057    float64
 57  attr058    float64
 58  attr059    float64
 59  attr060    float64
 60  attr061    float64
 61  attr062    float64
 62  attr063    float64
 63  attr064    float64
 64  attr065    float64
 65  attr066    float64
 66  attr067    float64
 67  attr068    float64
 68  attr069    float64
 69  attr070    float64
 70  attr071    float64
 71  attr072    float64
 72  attr073    float64
 73  attr074    float64
 74  attr075    float64
 75  attr076    float64
 76  attr077    float64
 77  attr078    float64
 78  attr079    float64
 79  attr080    float64
 80  attr081    float64
 81  attr082    float64
 82  attr083    float64
 83  attr084    float64
 84  attr085    float64
 85  attr086    float64
 86  attr087    float64
 87  attr088    float64
 88  attr089    float64
 89  attr090    float64
 90  attr091    float64
 91  attr092    float64
 92  attr093    float64
 93  attr094    float64
 94  attr095    float64
 95  attr096    float64
 96  attr097    float64
 97  attr098    float64
 98  attr099    float64
 99  attr100    float64
 100 attr101    float64
 101 attr102    float64
 102 attr103    float64
 103 attr104    float64
 104 attr105    float64
 105 attr106    float64
 106 attr107    float64
 107 attr108    float64
 108 attr109    float64
 109 attr110    float64
 110 attr111    float64
 111 attr112    float64
 112 attr113    float64
 113 attr114    float64
 114 attr115    float64
 115 attr116    float64
 116 attr117    float64
 117 attr118    float64
 118 attr119    float64
 119 attr120    float64
 120 attr121    float64
 121 attr122    float64
 122 attr123    float64
 123 attr124    float64
 124 attr125    float64
 125 attr126    float64
 126 attr127    float64
 127 attr128    float64
 128 attr129    float64
 129 attr130    float64
 130 attr131    float64
 131 attr132    float64
 132 attr133    float64
 133 attr134    float64
 134 attr135    float64
 135 attr136    float64
 136 attr137    float64
 137 attr138    float64
 138 attr139    float64
 139 attr140    float64
 140 attr141    float64
 141 attr142    float64
 142 attr143    float64
 143 attr144    float64
 144 attr145    float64
 145 attr146    float64
 146 attr147    float64
 147 attr148    float64
 148 attr149    float64
 149 attr150    float64
 150 attr151    float64
 151 attr152    float64
 152 attr153    float64
 153 attr154    float64
 154 attr155    float64
 155 attr156    float64
 156 attr157    float64
 157 attr158    float64
 158 attr159    float64
 159 attr160    float64
 160 attr161    float64
 161 attr162    float64
 162 attr163    float64
 163 attr164    float64
 164 attr165    float64
 165 attr166    float64
 166 attr167    float64
 167 attr168    float64
 168 attr169    float64
 169 attr170    float64
 170 attr171    float64
 171 attr172    float64
 172 attr173    float64
 173 attr174    float64
 174 attr175    float64
 175 attr176    float64
 176 attr177    float64
 177 attr178    float64
 178 attr179    float64
 179 attr180    float64
 180 attr181    float64
 181 attr182    float64
 182 attr183    float64
 183 attr184    float64
 184 attr185    float64
 185 attr186    float64
 186 attr187    float64
 187 attr188    float64
 188 attr189    float64
 189 attr190    float64
 190 attr191    float64
 191 attr192    float64
 192 attr193    float64
 193 attr194    float64
 194 attr195    float64
 195 attr196    float64
 196 attr197    float64
 197 attr198    float64
 198 attr199    float64
 199 attr200    float64
 200 attr201    float64
 201 attr202    float64
 202 attr203    float64
 203 attr204    float64
 204 attr205    float64
 205 attr206    float64
 206 attr207    float64
 207 attr208    float64
 208 attr209    float64
 209 attr210    float64
 210 attr211    float64
 211 attr212    float64
 212 attr213    float64
 213 attr214    float64
 214 attr215    float64
 215 attr216    float64
 216 attr217    float64
 217 attr218    float64
 218 attr219    float64
 219 attr220    float64
 220 attr221    float64
 221 attr222    float64
 222 attr223    float64
 223 attr224    float64
 224 attr225    float64
 225 attr226    float64
 226 attr227    float64
 227 attr228    float64
 228 attr229    float64
 229 attr230    float64
 230 attr231    float64
 231 attr232    float64
 232 attr233    float64
 233 attr234    float64
 234 attr235    float64
 235 attr236    float64
 236 attr237    float64
 237 attr238    float64
 238 attr239    float64
 239 attr240    float64
 240 attr241    float64
 241 attr242    float64
 242 attr243    float64
 243 attr244    float64
 244 attr245    float64
 245 attr246    float64
 246 attr247    float64
 247 attr248    float64
 248 attr249    float64
 249 attr250    float64
 250 attr251    float64
 251 attr252    float64
 252 attr253    float64
 253 attr254    float64
 254 attr255    float64
 255 attr256    float64
 256 attr257    float64
 257 attr258    float64
 258 attr259    float64
 259 attr260    float64
 260 attr261    float64
 261 attr262    float64
 262 attr263    float64
 263 attr264    float64
 264 attr265    float64
 265 attr266    float64
 266 attr267    float64
 267 attr268    float64
 268 attr269    float64
 269 attr270    float64
 270 attr271    float64
 271 attr272    float64
 272 attr273    float64
 273 attr274    float64
 274 attr275    float64
 275 attr276    float64
 276 attr277    float64
 277 attr278    float64
 278 attr279    float64
 279 attr280    float64
 280 attr281    float64
 281 attr282    float64
 282 attr283    float64
 283 attr284    float64
 284 attr285    float64
 285 attr286    float64
 286 attr287    float64
 287 attr288    float64
 288 attr289    float64
 289 attr290    float64
 290 attr291    float64
 291 attr292    float64
 292 attr293    float64
 293 attr294    float64
 294 attr295    float64
 295 attr296    float64
 296 attr297    float64
 297 attr298    float64
 298 attr299    float64
 299 attr300    float64
 300 attr301    float64
 301 attr302    float64
 302 attr303    float64
 303 attr304    float64
 304 attr305    float64
 305 attr306    float64
 306 attr307    float64
 307 attr308    float64
 308 attr309    float64
 309 attr310    float64
 310 attr311    float64
 311 attr312    float64
 312 attr313    float64
 313 attr314    float64
 314 attr315    float64
 315 attr316    float64
 316 attr317    float64
 317 attr318    float64
 318 attr319    float64
 319 attr320    float64
 320 attr321    float64
 321 attr322    float64
 322 attr323    float64
 323 attr324    float64
 324 attr325    float64
 325 attr326    float64
 326 attr327    float64
 327 attr328    float64
 328 attr329    float64
 329 attr330    float64
 330 attr331    float64
 331 attr332    float64
 332 attr333    float64
 333 attr334    float64
 334 attr335    float64
 335 attr336    float64
 336 attr337    float64
 337 attr338    float64
 338 attr339    float64
 339 attr340    float64
 340 attr341    float64
 341 attr342    float64
 342 attr343    float64
 343 attr344    float64
 344 attr345    float64
 345 attr346    float64
 346 attr347    float64
 347 attr348    float64
 348 attr349    float64
 349 attr350    float64
 350 attr351    float64
 351 attr352    float64
 352 attr353    float64
 353 attr354    float64
 354 attr355    float64
 355 attr356    float64
 356 attr357    float64
 357 attr358    float64
 358 attr359    float64
 359 attr360    float64
 360 attr361    float64
 361 attr362    float64
 362 attr363    float64
 363 attr364    float64
 364 attr365    float64
 365 attr366    float64
 366 attr367    float64
 367 attr368    float64
 368 attr369    float64
 369 attr370    float64
 370 attr371    float64
 371 attr372    float64
 372 attr373    float64
 373 attr374    float64
 374 attr375    float64
 375 attr376    float64
 376 attr377    float64
 377 attr378    float64
 378 attr379    float64
 379 attr380    float64
 380 attr381    float64
 381 attr382    float64
 382 attr383    float64
 383 attr384    float64
 384 attr385    float64
 385 attr386    float64
 386 attr387    float64
 387 attr388    float64
 388 attr389    float64
 389 attr390    float64
 390 attr391    float64
 391 attr392    float64
 392 attr393    float64
 393 attr394    float64
 394 attr395    float64
 395 attr396    float64
 396 attr397    float64
 397 attr398    float64
 398 attr399    float64
 399 attr400    float64
 400 attr401    float64
 401 attr402    float64
 402 attr403    float64
 403 attr404    float64
 404 attr405    float64
 405 attr406    float64
 406 attr407    float64
 407 attr408    float64
 408 attr409    float64
 409 attr410    float64
 410 attr411    float64
 411 attr412    float64
 412 attr413    float64
 413 attr414    float64
 414 attr415    float64
 415 attr416    float64
 416 attr417    float64
 417 attr418    float64
 418 attr419    float64
 419 attr420    float64
 420 attr421    float64
 421 attr422    float64
 422 attr423    float64
 423 attr424    float64
 424 attr425    float64
 425 attr426    float64
 426 attr427    float64
 427 attr428    float64
 428 attr429    float64
 429 attr430    float64
 430 attr431    float64
 431 attr432    float64
 432 attr433    float64
 433 attr434    float64
 434 attr435    float64
 435 attr436    float64
 436 attr437    float64
 437 attr438    float64
 438 attr439    float64
 439 attr440    float64
 440 attr441    float64
 441 attr442    float64
 442 attr443    float64
 443 attr444    float64
 444 attr445    float64
 445 attr446    float64
 446 attr447    float64
 447 attr448    float64
 448 attr449    float64
 449 attr450    float64
 450 attr451    float64
 451 attr452    float64
 452 attr453    float64
 453 attr454    float64
 454 attr455    float64
 455 attr456    float64
 456 attr457    float64
 457 attr458    float64
 458 attr459    float64
 459 attr460    float64
 460 attr461    float64
 461 attr462    float64
 462 attr463    float64
 463 attr464    float64
 464 attr465    float64
 465 attr466    float64
 466 attr467    float64
 467 attr468    float64
 468 attr469    float64
 469 attr470    float64
 470 attr471    float64
 471 attr472    float64
 472 attr473    float64
 473 attr474    float64
 474 attr475    float64
 475 attr476    float64
 476 attr477    float64
 477 attr478    float64
 478 attr479    float64
 479 attr480    float64
 480 attr481    float64
 481 attr482    float64
 482 attr483    float64
 483 attr484    float64
 484 attr485    float64
 485 attr486    float64
 486 attr487    float64
 487 attr488    float64
 488 attr489    float64
 489 attr490    float64
 490 attr491    float64
 491 attr492    float64
 492 attr493    float64
 493 attr494    float64
 494 attr495    float64
 495 attr496    float64
 496 attr497    float64
 497 attr498    float64
 498 attr499    float64
 499 attr500    float64
 500 attr501    float64
 501 attr502    float64
 502 attr503    float64
 503 attr504    float64
 504 attr505    float64
 505 attr506    float64
 506 attr507    float64
 507 attr508    float64
 508 attr509    float64
 509 attr510    float64
 510 attr511    float64
 511 attr512    float64
 512 attr513    float64
 513 attr514    float64
 514 attr515    float64
 515 attr516    float64
 516 attr517    float64
 517 attr518    float64
 518 attr519    float64
 519 attr520    float64
 520 attr521    float64
 521 attr522    float64
 522 attr523    float64
 523 attr524    float64
 524 attr525    float64
 525 attr526    float64
 526 attr527    float64
 527 attr528    float64
 528 attr529    float64
 529 attr530    float64
 530 attr531    float64
 531 attr532    float64
 532 attr533    float64
 533 attr534    float64
 534 attr535    float64
 535 attr536    float64
 536 attr537    float64
 537 attr538    float64
 538 attr539    float64
 539 attr540    float64
 540 attr541    float64
 541 attr542    float64
 542 attr543    float64
 543 attr544    float64
 544 attr545    float64
 545 attr546    float64
 546 attr547    float64
 547 attr548    float64
 548 attr549    float64
 549 attr550    float64
 550 attr551    float64
 551 attr552    float64
 552 attr553    float64
 553 attr554    float64
 554 attr555    float64
 555 attr556    float64
 556 attr557    float64
 557 attr558    float64
 558 attr559    float64
 559 attr560    float64
 560 attr561    float64
 561 targetVar  int64  
dtypes: float64(561), int64(1)
memory usage: 44.2 MB

Xy_original.describe()

	attr001	attr002	attr003	attr004	attr005	attr006	attr007	attr008	attr009	attr010	attr011	attr012	attr013	attr014	attr015	attr016	attr017	attr018	attr019	attr020	attr021	attr022	attr023	attr024	attr025	attr026	attr027	attr028	attr029	attr030	attr031	attr032	attr033	attr034	attr035	attr036	attr037	attr038	attr039	attr040	...	attr523	attr524	attr525	attr526	attr527	attr528	attr529	attr530	attr531	attr532	attr533	attr534	attr535	attr536	attr537	attr538	attr539	attr540	attr541	attr542	attr543	attr544	attr545	attr546	attr547	attr548	attr549	attr550	attr551	attr552	attr553	attr554	attr555	attr556	attr557	attr558	attr559	attr560	attr561	targetVar
count	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	...	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000	10299.000000
mean	0.274347	-0.017743	-0.108925	-0.607784	-0.510191	-0.613064	-0.633593	-0.525697	-0.614989	-0.466732	-0.305180	-0.562230	0.525304	0.389537	0.598022	-0.552087	-0.825460	-0.902704	-0.854662	-0.689162	-0.643512	-0.640686	-0.100332	-0.128765	-0.157863	-0.118954	0.108574	-0.035699	0.122000	-0.029677	0.031724	0.155148	-0.018077	0.006110	0.037729	0.034424	-0.082669	-0.120309	-0.197746	0.102199	...	-0.676629	-0.338469	-0.877800	0.173220	-0.298598	-0.601659	-0.697411	-0.699976	-0.681014	-0.734623	-0.888701	-0.697411	-0.881301	-0.722125	-0.076279	-0.886999	-0.041564	-0.264279	-0.575866	-0.779768	-0.792190	-0.773404	-0.809934	-0.871201	-0.779768	-0.937898	-0.772715	-0.274339	-0.900033	0.126708	-0.298592	-0.617700	0.007705	0.002648	0.017683	-0.009219	-0.496522	0.063255	-0.054284	3.624624
std	0.067628	0.037128	0.053033	0.438694	0.500240	0.403657	0.413333	0.484201	0.399034	0.538707	0.279920	0.282991	0.356589	0.338844	0.290615	0.461375	0.247052	0.125988	0.205998	0.359209	0.368865	0.372065	0.462315	0.433687	0.368102	0.307721	0.247140	0.247820	0.232044	0.254461	0.213708	0.208718	0.220257	0.281722	0.215456	0.236854	0.230957	0.357677	0.325103	0.376049	...	0.365550	0.666857	0.189655	0.252537	0.364723	0.353928	0.323701	0.310443	0.331222	0.281107	0.163587	0.323701	0.180432	0.310562	0.602930	0.159981	0.280142	0.322579	0.320006	0.267592	0.259160	0.279727	0.242697	0.189627	0.267592	0.128284	0.278403	0.624272	0.139691	0.245443	0.320199	0.308796	0.336591	0.447364	0.616188	0.484770	0.511158	0.305468	0.268898	1.743695
min	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	...	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	1.000000
25%	0.262625	-0.024902	-0.121019	-0.992360	-0.976990	-0.979137	-0.993293	-0.977017	-0.979064	-0.935788	-0.562570	-0.812194	0.212530	0.113900	0.392717	-0.981706	-0.999929	-0.999771	-0.999414	-0.994185	-0.981327	-0.978488	-0.563834	-0.549611	-0.496825	-0.368555	-0.079017	-0.189949	-0.033860	-0.221971	-0.129050	0.028955	-0.165678	-0.206485	-0.118064	-0.110770	-0.239535	-0.361669	-0.408798	-0.140909	...	-0.988546	-1.000000	-0.968254	-0.002959	-0.601439	-0.878842	-0.982502	-0.978150	-0.978668	-0.979648	-0.993815	-0.982502	-0.999668	-0.985373	-0.665001	-1.000000	-0.234356	-0.499586	-0.807658	-0.992108	-0.992575	-0.991675	-0.993546	-0.993657	-0.992108	-0.999946	-0.991164	-0.923452	-0.968254	-0.019481	-0.536174	-0.841847	-0.124694	-0.287031	-0.493108	-0.389041	-0.817288	0.002151	-0.131880	2.000000
50%	0.277174	-0.017162	-0.108596	-0.943030	-0.835032	-0.850773	-0.948244	-0.843670	-0.845068	-0.874825	-0.468206	-0.724503	0.784233	0.619774	0.772226	-0.876947	-0.997736	-0.992909	-0.984240	-0.955999	-0.884947	-0.853776	-0.057117	-0.101741	-0.136380	-0.136206	0.077529	-0.017644	0.126279	-0.045486	0.017654	0.160661	-0.018927	0.020698	0.009945	0.045355	-0.083301	-0.161167	-0.191758	0.135572	...	-0.942217	-0.682105	-0.904762	0.164180	-0.347522	-0.713718	-0.875620	-0.827490	-0.845626	-0.827234	-0.959151	-0.875620	-0.984283	-0.912569	-0.155022	-0.948718	-0.052095	-0.317706	-0.664937	-0.945344	-0.938212	-0.935101	-0.943402	-0.972651	-0.945344	-0.998043	-0.941888	-0.414503	-0.904762	0.136245	-0.335160	-0.703402	0.008146	0.007668	0.017192	-0.007186	-0.715631	0.182028	-0.003882	4.000000
75%	0.288354	-0.010625	-0.097589	-0.250293	-0.057336	-0.278737	-0.302033	-0.087405	-0.288149	-0.014641	-0.067345	-0.345591	0.843793	0.685194	0.836742	-0.122829	-0.715745	-0.825149	-0.759473	-0.407902	-0.324653	-0.336393	0.329587	0.283141	0.167352	0.133245	0.286068	0.133317	0.277676	0.163282	0.180850	0.288163	0.131190	0.223530	0.179591	0.194290	0.074748	0.080148	0.002491	0.372200	...	-0.371697	0.346184	-0.873016	0.357307	-0.057690	-0.425813	-0.451400	-0.471267	-0.418532	-0.555974	-0.839875	-0.451400	-0.814917	-0.495328	0.513909	-0.846154	0.151575	-0.084974	-0.439280	-0.612242	-0.643738	-0.609846	-0.684886	-0.805758	-0.612242	-0.922682	-0.604730	0.337220	-0.873016	0.288960	-0.113167	-0.487981	0.149005	0.291490	0.536137	0.365996	-0.521503	0.250790	0.102970	5.000000
max	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	...	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	6.000000

8 rows × 562 columns

Xy_original.isnull().sum()

attr001      0
attr002      0
attr003      0
attr004      0
attr005      0
            ..
attr558      0
attr559      0
attr560      0
attr561      0
targetVar    0
Length: 562, dtype: int64

print('Total number of NaN in the dataframe: ', Xy_original.isnull().sum().sum())

Total number of NaN in the dataframe:  0

1.d) Data Cleaning

# Not applicable for this iteration of the project

1.e) Set up the parameters for data visualization

# Use variable totCol to hold the number of columns in the dataframe
totCol = len(Xy_original.columns)

# Set up variable totAttr for the total number of attribute columns
totAttr = totCol-1

# targetCol variable indicates the column location of the target/class variable
# If the first column, set targetCol to 1. If the last column, set targetCol to totCol
# If (targetCol <> 1) and (targetCol <> totCol), be aware when slicing up the dataframes for visualization
targetCol = totCol

# Set up the number of row and columns for visualization display. dispRow * dispCol should be >= totAttr
dispCol = 4
if totAttr % dispCol == 0 :
    dispRow = totAttr // dispCol
else :
    dispRow = (totAttr // dispCol) + 1
    
# Set figure width to display the data visualization plots
fig_size = plt.rcParams["figure.figsize"]
fig_size[0] = dispCol*4
fig_size[1] = dispRow*4
plt.rcParams["figure.figsize"] = fig_size

Task 2. Summarize Data

2.a) Descriptive Statistics

2.a.i) Peek at the attribute columns

X_train_df.head(10)

	attr001	attr002	attr003	attr004	attr005	attr006	attr007	attr008	attr009	attr010	attr011	attr012	attr013	attr014	attr015	attr016	attr017	attr018	attr019	attr020	attr021	attr022	attr023	attr024	attr025	attr026	attr027	attr028	attr029	attr030	attr031	attr032	attr033	attr034	attr035	attr036	attr037	attr038	attr039	attr040	...	attr522	attr523	attr524	attr525	attr526	attr527	attr528	attr529	attr530	attr531	attr532	attr533	attr534	attr535	attr536	attr537	attr538	attr539	attr540	attr541	attr542	attr543	attr544	attr545	attr546	attr547	attr548	attr549	attr550	attr551	attr552	attr553	attr554	attr555	attr556	attr557	attr558	attr559	attr560	attr561
0	0.288585	-0.020294	-0.132905	-0.995279	-0.983111	-0.913526	-0.995112	-0.983185	-0.923527	-0.934724	-0.567378	-0.744413	0.852947	0.685845	0.814263	-0.965523	-0.999945	-0.999863	-0.994612	-0.994231	-0.987614	-0.943220	-0.407747	-0.679338	-0.602122	0.929294	-0.853011	0.359910	-0.058526	0.256892	-0.224848	0.264106	-0.095246	0.278851	-0.465085	0.491936	-0.190884	0.376314	0.435129	0.660790	...	-0.999918	-0.991364	-1.0	-0.936508	0.346989	-0.516080	-0.802760	-0.980135	-0.961309	-0.973653	-0.952264	-0.989498	-0.980135	-0.999240	-0.992656	-0.701291	-1.000000	-0.128989	0.586156	0.374605	-0.991990	-0.990697	-0.989941	-0.992448	-0.991048	-0.991990	-0.999937	-0.990458	-0.871306	-1.000000	-0.074323	-0.298676	-0.710304	-0.112754	0.030400	-0.464761	-0.018446	-0.841247	0.179941	-0.058627
1	0.278419	-0.016411	-0.123520	-0.998245	-0.975300	-0.960322	-0.998807	-0.974914	-0.957686	-0.943068	-0.557851	-0.818409	0.849308	0.685845	0.822637	-0.981930	-0.999991	-0.999788	-0.998405	-0.999150	-0.977866	-0.948225	-0.714892	-0.500930	-0.570979	0.611627	-0.329549	0.284213	0.284595	0.115705	-0.090963	0.294310	-0.281211	0.085988	-0.022153	-0.016657	-0.220643	-0.013429	-0.072692	0.579382	...	-0.999867	-0.991134	-1.0	-0.841270	0.532061	-0.624871	-0.900160	-0.988296	-0.983322	-0.982659	-0.986321	-0.991829	-0.988296	-0.999811	-0.993979	-0.720683	-0.948718	-0.271958	-0.336310	-0.720015	-0.995854	-0.996399	-0.995442	-0.996866	-0.994440	-0.995854	-0.999981	-0.994544	-1.000000	-1.000000	0.158075	-0.595051	-0.861499	0.053477	-0.007435	-0.732626	0.703511	-0.844788	0.180289	-0.054317
2	0.279653	-0.019467	-0.113462	-0.995380	-0.967187	-0.978944	-0.996520	-0.963668	-0.977469	-0.938692	-0.557851	-0.818409	0.843609	0.682401	0.839344	-0.983478	-0.999969	-0.999660	-0.999470	-0.997130	-0.964810	-0.974675	-0.592235	-0.485821	-0.570979	0.273025	-0.086309	0.337202	-0.164739	0.017150	-0.074507	0.342256	-0.332564	0.239281	-0.136204	0.173863	-0.299493	-0.124698	-0.181105	0.608900	...	-0.999845	-0.986658	-1.0	-0.904762	0.660795	-0.724697	-0.928539	-0.989255	-0.986028	-0.984274	-0.990979	-0.995703	-0.989255	-0.999854	-0.993238	-0.736521	-0.794872	-0.212728	-0.535352	-0.871914	-0.995031	-0.995127	-0.994640	-0.996060	-0.995866	-0.995031	-0.999973	-0.993755	-1.000000	-0.555556	0.414503	-0.390748	-0.760104	-0.118559	0.177899	0.100699	0.808529	-0.848933	0.180637	-0.049118
3	0.279174	-0.026201	-0.123283	-0.996091	-0.983403	-0.990675	-0.997099	-0.982750	-0.989302	-0.938692	-0.576159	-0.829711	0.843609	0.682401	0.837869	-0.986093	-0.999976	-0.999736	-0.999504	-0.997180	-0.983799	-0.986007	-0.627446	-0.850930	-0.911872	0.061436	0.074840	0.198204	-0.264307	0.072545	-0.155320	0.323154	-0.170813	0.294938	-0.306081	0.482148	-0.470129	-0.305693	-0.362654	0.507459	...	-0.999895	-0.988055	-1.0	1.000000	0.678921	-0.701131	-0.909639	-0.989413	-0.987836	-0.986850	-0.986749	-0.996199	-0.989413	-0.999876	-0.989136	-0.720891	-1.000000	-0.035684	-0.230091	-0.511217	-0.995221	-0.995237	-0.995722	-0.995273	-0.995732	-0.995221	-0.999974	-0.995226	-0.955696	-0.936508	0.404573	-0.117290	-0.482845	-0.036788	-0.012892	0.640011	-0.485366	-0.848649	0.181935	-0.047663
4	0.276629	-0.016570	-0.115362	-0.998139	-0.980817	-0.990482	-0.998321	-0.979672	-0.990441	-0.942469	-0.569174	-0.824705	0.849095	0.683250	0.837869	-0.992653	-0.999991	-0.999856	-0.999757	-0.998004	-0.981232	-0.991325	-0.786553	-0.559477	-0.761434	0.313276	-0.131208	0.191161	0.086904	0.257615	-0.272505	0.434728	-0.315375	0.439744	-0.269069	0.179414	-0.088952	-0.155804	-0.189763	0.599213	...	-0.999941	-0.994169	-1.0	-1.000000	0.559058	-0.528901	-0.858933	-0.991433	-0.989059	-0.987744	-0.991462	-0.998353	-0.991433	-0.999902	-0.989321	-0.763372	-0.897436	-0.273582	-0.510282	-0.830702	-0.995093	-0.995465	-0.995279	-0.995609	-0.997418	-0.995093	-0.999974	-0.995487	-1.000000	-0.936508	0.087753	-0.351471	-0.699205	0.123320	0.122542	0.693578	-0.615971	-0.847865	0.185151	-0.043892
5	0.277199	-0.010098	-0.105137	-0.997335	-0.990487	-0.995420	-0.997627	-0.990218	-0.995549	-0.942469	-0.565684	-0.822766	0.849095	0.695586	0.845922	-0.993928	-0.999985	-0.999857	-0.999917	-0.997584	-0.991847	-0.995414	-0.751869	-0.454773	-0.550882	0.390052	-0.182272	0.158751	0.187313	0.259951	-0.243230	0.421736	-0.418460	0.558461	-0.218344	0.165138	0.080920	-0.209979	-0.151064	0.180424	...	-0.999937	-0.996697	-1.0	-1.000000	0.246910	-0.520879	-0.802525	-0.990500	-0.985861	-0.985512	-0.988204	-0.994758	-0.990500	-0.999861	-0.991967	-0.768577	-1.000000	-0.297329	-0.346045	-0.727270	-0.995143	-0.995239	-0.994398	-0.996088	-0.998519	-0.995143	-0.999974	-0.994530	-1.000000	-1.000000	0.019953	-0.545410	-0.844619	0.082632	-0.143439	0.275041	-0.368224	-0.849632	0.184823	-0.042126
6	0.279454	-0.019641	-0.110022	-0.996921	-0.967186	-0.983118	-0.997003	-0.966097	-0.983116	-0.940987	-0.565684	-0.817189	0.851040	0.674347	0.833591	-0.986837	-0.999982	-0.999658	-0.999634	-0.996863	-0.972402	-0.982864	-0.637185	-0.514751	-0.536553	0.360460	-0.233335	0.226457	0.069525	0.064297	-0.076439	0.138057	-0.036769	0.231401	-0.114577	0.319204	-0.487769	-0.095852	-0.135729	0.610845	...	-0.999824	-0.991345	-1.0	-0.904762	0.290177	-0.668835	-0.933443	-0.988269	-0.984569	-0.983601	-0.985328	-0.994521	-0.988269	-0.999828	-0.989040	-0.735639	-1.000000	-0.257032	-0.321591	-0.658435	-0.995641	-0.994639	-0.994202	-0.994491	-0.996934	-0.995641	-0.999974	-0.993940	-0.955696	-1.000000	0.145844	-0.217198	-0.564430	-0.212754	-0.230622	0.014637	-0.189512	-0.852150	0.182170	-0.043010
7	0.277432	-0.030488	-0.125360	-0.996559	-0.966728	-0.981585	-0.996485	-0.966313	-0.982982	-0.940987	-0.572638	-0.817189	0.850328	0.670410	0.832383	-0.976851	-0.999980	-0.999341	-0.999164	-0.995768	-0.975847	-0.986829	-0.633239	-0.743877	-0.822408	0.363064	-0.300719	0.346867	-0.063694	0.144124	-0.132349	0.083232	0.041631	0.137192	-0.092290	0.372369	-0.720913	-0.161634	-0.017120	0.561421	...	-0.999845	-0.989676	-1.0	-0.968254	0.249799	-0.655503	-0.932563	-0.989431	-0.987065	-0.985978	-0.988193	-0.998240	-0.989431	-0.999867	-0.990035	-0.735639	-1.000000	-0.197272	-0.407284	-0.733313	-0.995629	-0.994507	-0.994727	-0.994018	-0.997006	-0.995629	-0.999974	-0.995423	-0.955696	-1.000000	0.136382	-0.082307	-0.421715	-0.020888	0.593996	-0.561871	0.467383	-0.851017	0.183779	-0.041976
8	0.277293	-0.021751	-0.120751	-0.997328	-0.961245	-0.983672	-0.997596	-0.957236	-0.984379	-0.940598	-0.564175	-0.823527	0.850328	0.670410	0.832383	-0.983295	-0.999985	-0.999528	-0.999426	-0.997351	-0.957212	-0.985841	-0.683117	-0.524812	-0.758524	0.358268	-0.253309	0.266929	-0.124250	0.215324	-0.183362	0.171745	-0.068957	0.241831	-0.194419	0.216758	-0.338026	-0.164791	-0.033767	0.724234	...	-0.999894	-0.992371	-1.0	-1.000000	0.272342	-0.750485	-0.929474	-0.991629	-0.991759	-0.991182	-0.991522	-0.995197	-0.991629	-0.999927	-0.991428	-0.850310	-1.000000	0.073417	-0.371389	-0.674770	-0.995260	-0.996007	-0.995828	-0.995482	-0.993608	-0.995260	-0.999977	-0.995095	-0.955696	-1.000000	0.314038	-0.269401	-0.572995	0.012954	0.080936	-0.234313	0.117797	-0.847971	0.188982	-0.037364
9	0.280586	-0.009960	-0.106065	-0.994803	-0.972758	-0.986244	-0.995405	-0.973663	-0.985642	-0.940028	-0.554594	-0.815850	0.845442	0.684757	0.838455	-0.986541	-0.999963	-0.999662	-0.999718	-0.996163	-0.979809	-0.982811	-0.550721	-0.294553	-0.479711	0.192863	-0.021212	0.037915	0.190540	0.039722	0.054283	0.037268	0.081551	0.172340	0.035995	-0.184930	0.351156	-0.176241	-0.447651	0.391867	...	-0.999680	-0.979144	-1.0	-1.000000	0.250884	-0.510116	-0.801919	-0.984788	-0.980388	-0.981850	-0.976771	-0.993197	-0.984788	-0.999736	-0.981969	-0.671366	-1.000000	-0.139966	0.106785	-0.131360	-0.990981	-0.990599	-0.991665	-0.988199	-0.984877	-0.990981	-0.999929	-0.990456	-0.955696	-1.000000	0.267383	0.339526	0.140452	-0.020590	-0.127730	-0.482871	-0.070670	-0.848294	0.190310	-0.034417

10 rows × 561 columns

2.a.ii) Dimensions and attribute types

X_train_df.info(verbose=True)

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 7352 entries, 0 to 7351
Data columns (total 561 columns):
 #   Column   Dtype  
---  ------   -----  
 0   attr001  float64
 1   attr002  float64
 2   attr003  float64
 3   attr004  float64
 4   attr005  float64
 5   attr006  float64
 6   attr007  float64
 7   attr008  float64
 8   attr009  float64
 9   attr010  float64
 10  attr011  float64
 11  attr012  float64
 12  attr013  float64
 13  attr014  float64
 14  attr015  float64
 15  attr016  float64
 16  attr017  float64
 17  attr018  float64
 18  attr019  float64
 19  attr020  float64
 20  attr021  float64
 21  attr022  float64
 22  attr023  float64
 23  attr024  float64
 24  attr025  float64
 25  attr026  float64
 26  attr027  float64
 27  attr028  float64
 28  attr029  float64
 29  attr030  float64
 30  attr031  float64
 31  attr032  float64
 32  attr033  float64
 33  attr034  float64
 34  attr035  float64
 35  attr036  float64
 36  attr037  float64
 37  attr038  float64
 38  attr039  float64
 39  attr040  float64
 40  attr041  float64
 41  attr042  float64
 42  attr043  float64
 43  attr044  float64
 44  attr045  float64
 45  attr046  float64
 46  attr047  float64
 47  attr048  float64
 48  attr049  float64
 49  attr050  float64
 50  attr051  float64
 51  attr052  float64
 52  attr053  float64
 53  attr054  float64
 54  attr055  float64
 55  attr056  float64
 56  attr057  float64
 57  attr058  float64
 58  attr059  float64
 59  attr060  float64
 60  attr061  float64
 61  attr062  float64
 62  attr063  float64
 63  attr064  float64
 64  attr065  float64
 65  attr066  float64
 66  attr067  float64
 67  attr068  float64
 68  attr069  float64
 69  attr070  float64
 70  attr071  float64
 71  attr072  float64
 72  attr073  float64
 73  attr074  float64
 74  attr075  float64
 75  attr076  float64
 76  attr077  float64
 77  attr078  float64
 78  attr079  float64
 79  attr080  float64
 80  attr081  float64
 81  attr082  float64
 82  attr083  float64
 83  attr084  float64
 84  attr085  float64
 85  attr086  float64
 86  attr087  float64
 87  attr088  float64
 88  attr089  float64
 89  attr090  float64
 90  attr091  float64
 91  attr092  float64
 92  attr093  float64
 93  attr094  float64
 94  attr095  float64
 95  attr096  float64
 96  attr097  float64
 97  attr098  float64
 98  attr099  float64
 99  attr100  float64
 100 attr101  float64
 101 attr102  float64
 102 attr103  float64
 103 attr104  float64
 104 attr105  float64
 105 attr106  float64
 106 attr107  float64
 107 attr108  float64
 108 attr109  float64
 109 attr110  float64
 110 attr111  float64
 111 attr112  float64
 112 attr113  float64
 113 attr114  float64
 114 attr115  float64
 115 attr116  float64
 116 attr117  float64
 117 attr118  float64
 118 attr119  float64
 119 attr120  float64
 120 attr121  float64
 121 attr122  float64
 122 attr123  float64
 123 attr124  float64
 124 attr125  float64
 125 attr126  float64
 126 attr127  float64
 127 attr128  float64
 128 attr129  float64
 129 attr130  float64
 130 attr131  float64
 131 attr132  float64
 132 attr133  float64
 133 attr134  float64
 134 attr135  float64
 135 attr136  float64
 136 attr137  float64
 137 attr138  float64
 138 attr139  float64
 139 attr140  float64
 140 attr141  float64
 141 attr142  float64
 142 attr143  float64
 143 attr144  float64
 144 attr145  float64
 145 attr146  float64
 146 attr147  float64
 147 attr148  float64
 148 attr149  float64
 149 attr150  float64
 150 attr151  float64
 151 attr152  float64
 152 attr153  float64
 153 attr154  float64
 154 attr155  float64
 155 attr156  float64
 156 attr157  float64
 157 attr158  float64
 158 attr159  float64
 159 attr160  float64
 160 attr161  float64
 161 attr162  float64
 162 attr163  float64
 163 attr164  float64
 164 attr165  float64
 165 attr166  float64
 166 attr167  float64
 167 attr168  float64
 168 attr169  float64
 169 attr170  float64
 170 attr171  float64
 171 attr172  float64
 172 attr173  float64
 173 attr174  float64
 174 attr175  float64
 175 attr176  float64
 176 attr177  float64
 177 attr178  float64
 178 attr179  float64
 179 attr180  float64
 180 attr181  float64
 181 attr182  float64
 182 attr183  float64
 183 attr184  float64
 184 attr185  float64
 185 attr186  float64
 186 attr187  float64
 187 attr188  float64
 188 attr189  float64
 189 attr190  float64
 190 attr191  float64
 191 attr192  float64
 192 attr193  float64
 193 attr194  float64
 194 attr195  float64
 195 attr196  float64
 196 attr197  float64
 197 attr198  float64
 198 attr199  float64
 199 attr200  float64
 200 attr201  float64
 201 attr202  float64
 202 attr203  float64
 203 attr204  float64
 204 attr205  float64
 205 attr206  float64
 206 attr207  float64
 207 attr208  float64
 208 attr209  float64
 209 attr210  float64
 210 attr211  float64
 211 attr212  float64
 212 attr213  float64
 213 attr214  float64
 214 attr215  float64
 215 attr216  float64
 216 attr217  float64
 217 attr218  float64
 218 attr219  float64
 219 attr220  float64
 220 attr221  float64
 221 attr222  float64
 222 attr223  float64
 223 attr224  float64
 224 attr225  float64
 225 attr226  float64
 226 attr227  float64
 227 attr228  float64
 228 attr229  float64
 229 attr230  float64
 230 attr231  float64
 231 attr232  float64
 232 attr233  float64
 233 attr234  float64
 234 attr235  float64
 235 attr236  float64
 236 attr237  float64
 237 attr238  float64
 238 attr239  float64
 239 attr240  float64
 240 attr241  float64
 241 attr242  float64
 242 attr243  float64
 243 attr244  float64
 244 attr245  float64
 245 attr246  float64
 246 attr247  float64
 247 attr248  float64
 248 attr249  float64
 249 attr250  float64
 250 attr251  float64
 251 attr252  float64
 252 attr253  float64
 253 attr254  float64
 254 attr255  float64
 255 attr256  float64
 256 attr257  float64
 257 attr258  float64
 258 attr259  float64
 259 attr260  float64
 260 attr261  float64
 261 attr262  float64
 262 attr263  float64
 263 attr264  float64
 264 attr265  float64
 265 attr266  float64
 266 attr267  float64
 267 attr268  float64
 268 attr269  float64
 269 attr270  float64
 270 attr271  float64
 271 attr272  float64
 272 attr273  float64
 273 attr274  float64
 274 attr275  float64
 275 attr276  float64
 276 attr277  float64
 277 attr278  float64
 278 attr279  float64
 279 attr280  float64
 280 attr281  float64
 281 attr282  float64
 282 attr283  float64
 283 attr284  float64
 284 attr285  float64
 285 attr286  float64
 286 attr287  float64
 287 attr288  float64
 288 attr289  float64
 289 attr290  float64
 290 attr291  float64
 291 attr292  float64
 292 attr293  float64
 293 attr294  float64
 294 attr295  float64
 295 attr296  float64
 296 attr297  float64
 297 attr298  float64
 298 attr299  float64
 299 attr300  float64
 300 attr301  float64
 301 attr302  float64
 302 attr303  float64
 303 attr304  float64
 304 attr305  float64
 305 attr306  float64
 306 attr307  float64
 307 attr308  float64
 308 attr309  float64
 309 attr310  float64
 310 attr311  float64
 311 attr312  float64
 312 attr313  float64
 313 attr314  float64
 314 attr315  float64
 315 attr316  float64
 316 attr317  float64
 317 attr318  float64
 318 attr319  float64
 319 attr320  float64
 320 attr321  float64
 321 attr322  float64
 322 attr323  float64
 323 attr324  float64
 324 attr325  float64
 325 attr326  float64
 326 attr327  float64
 327 attr328  float64
 328 attr329  float64
 329 attr330  float64
 330 attr331  float64
 331 attr332  float64
 332 attr333  float64
 333 attr334  float64
 334 attr335  float64
 335 attr336  float64
 336 attr337  float64
 337 attr338  float64
 338 attr339  float64
 339 attr340  float64
 340 attr341  float64
 341 attr342  float64
 342 attr343  float64
 343 attr344  float64
 344 attr345  float64
 345 attr346  float64
 346 attr347  float64
 347 attr348  float64
 348 attr349  float64
 349 attr350  float64
 350 attr351  float64
 351 attr352  float64
 352 attr353  float64
 353 attr354  float64
 354 attr355  float64
 355 attr356  float64
 356 attr357  float64
 357 attr358  float64
 358 attr359  float64
 359 attr360  float64
 360 attr361  float64
 361 attr362  float64
 362 attr363  float64
 363 attr364  float64
 364 attr365  float64
 365 attr366  float64
 366 attr367  float64
 367 attr368  float64
 368 attr369  float64
 369 attr370  float64
 370 attr371  float64
 371 attr372  float64
 372 attr373  float64
 373 attr374  float64
 374 attr375  float64
 375 attr376  float64
 376 attr377  float64
 377 attr378  float64
 378 attr379  float64
 379 attr380  float64
 380 attr381  float64
 381 attr382  float64
 382 attr383  float64
 383 attr384  float64
 384 attr385  float64
 385 attr386  float64
 386 attr387  float64
 387 attr388  float64
 388 attr389  float64
 389 attr390  float64
 390 attr391  float64
 391 attr392  float64
 392 attr393  float64
 393 attr394  float64
 394 attr395  float64
 395 attr396  float64
 396 attr397  float64
 397 attr398  float64
 398 attr399  float64
 399 attr400  float64
 400 attr401  float64
 401 attr402  float64
 402 attr403  float64
 403 attr404  float64
 404 attr405  float64
 405 attr406  float64
 406 attr407  float64
 407 attr408  float64
 408 attr409  float64
 409 attr410  float64
 410 attr411  float64
 411 attr412  float64
 412 attr413  float64
 413 attr414  float64
 414 attr415  float64
 415 attr416  float64
 416 attr417  float64
 417 attr418  float64
 418 attr419  float64
 419 attr420  float64
 420 attr421  float64
 421 attr422  float64
 422 attr423  float64
 423 attr424  float64
 424 attr425  float64
 425 attr426  float64
 426 attr427  float64
 427 attr428  float64
 428 attr429  float64
 429 attr430  float64
 430 attr431  float64
 431 attr432  float64
 432 attr433  float64
 433 attr434  float64
 434 attr435  float64
 435 attr436  float64
 436 attr437  float64
 437 attr438  float64
 438 attr439  float64
 439 attr440  float64
 440 attr441  float64
 441 attr442  float64
 442 attr443  float64
 443 attr444  float64
 444 attr445  float64
 445 attr446  float64
 446 attr447  float64
 447 attr448  float64
 448 attr449  float64
 449 attr450  float64
 450 attr451  float64
 451 attr452  float64
 452 attr453  float64
 453 attr454  float64
 454 attr455  float64
 455 attr456  float64
 456 attr457  float64
 457 attr458  float64
 458 attr459  float64
 459 attr460  float64
 460 attr461  float64
 461 attr462  float64
 462 attr463  float64
 463 attr464  float64
 464 attr465  float64
 465 attr466  float64
 466 attr467  float64
 467 attr468  float64
 468 attr469  float64
 469 attr470  float64
 470 attr471  float64
 471 attr472  float64
 472 attr473  float64
 473 attr474  float64
 474 attr475  float64
 475 attr476  float64
 476 attr477  float64
 477 attr478  float64
 478 attr479  float64
 479 attr480  float64
 480 attr481  float64
 481 attr482  float64
 482 attr483  float64
 483 attr484  float64
 484 attr485  float64
 485 attr486  float64
 486 attr487  float64
 487 attr488  float64
 488 attr489  float64
 489 attr490  float64
 490 attr491  float64
 491 attr492  float64
 492 attr493  float64
 493 attr494  float64
 494 attr495  float64
 495 attr496  float64
 496 attr497  float64
 497 attr498  float64
 498 attr499  float64
 499 attr500  float64
 500 attr501  float64
 501 attr502  float64
 502 attr503  float64
 503 attr504  float64
 504 attr505  float64
 505 attr506  float64
 506 attr507  float64
 507 attr508  float64
 508 attr509  float64
 509 attr510  float64
 510 attr511  float64
 511 attr512  float64
 512 attr513  float64
 513 attr514  float64
 514 attr515  float64
 515 attr516  float64
 516 attr517  float64
 517 attr518  float64
 518 attr519  float64
 519 attr520  float64
 520 attr521  float64
 521 attr522  float64
 522 attr523  float64
 523 attr524  float64
 524 attr525  float64
 525 attr526  float64
 526 attr527  float64
 527 attr528  float64
 528 attr529  float64
 529 attr530  float64
 530 attr531  float64
 531 attr532  float64
 532 attr533  float64
 533 attr534  float64
 534 attr535  float64
 535 attr536  float64
 536 attr537  float64
 537 attr538  float64
 538 attr539  float64
 539 attr540  float64
 540 attr541  float64
 541 attr542  float64
 542 attr543  float64
 543 attr544  float64
 544 attr545  float64
 545 attr546  float64
 546 attr547  float64
 547 attr548  float64
 548 attr549  float64
 549 attr550  float64
 550 attr551  float64
 551 attr552  float64
 552 attr553  float64
 553 attr554  float64
 554 attr555  float64
 555 attr556  float64
 556 attr557  float64
 557 attr558  float64
 558 attr559  float64
 559 attr560  float64
 560 attr561  float64
dtypes: float64(561)
memory usage: 31.5 MB

2.a.iii) Statistical summary of the attributes

X_train_df.describe()

	attr001	attr002	attr003	attr004	attr005	attr006	attr007	attr008	attr009	attr010	attr011	attr012	attr013	attr014	attr015	attr016	attr017	attr018	attr019	attr020	attr021	attr022	attr023	attr024	attr025	attr026	attr027	attr028	attr029	attr030	attr031	attr032	attr033	attr034	attr035	attr036	attr037	attr038	attr039	attr040	...	attr522	attr523	attr524	attr525	attr526	attr527	attr528	attr529	attr530	attr531	attr532	attr533	attr534	attr535	attr536	attr537	attr538	attr539	attr540	attr541	attr542	attr543	attr544	attr545	attr546	attr547	attr548	attr549	attr550	attr551	attr552	attr553	attr554	attr555	attr556	attr557	attr558	attr559	attr560	attr561
count	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	...	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000	7352.000000
mean	0.274488	-0.017695	-0.109141	-0.605438	-0.510938	-0.604754	-0.630512	-0.526907	-0.606150	-0.468604	-0.306043	-0.557121	0.523551	0.387386	0.594374	-0.547569	-0.820041	-0.901874	-0.845784	-0.684345	-0.643770	-0.631069	-0.102993	-0.137937	-0.163946	-0.116599	0.102762	-0.037786	0.130477	-0.026229	0.026322	0.159966	-0.019575	0.009420	0.033291	0.036587	-0.078640	-0.125131	-0.193802	0.105005	...	-0.842188	-0.678618	-0.347657	-0.877995	0.178195	-0.312968	-0.615441	-0.693210	-0.692876	-0.674830	-0.726645	-0.885103	-0.693210	-0.874292	-0.719795	-0.087878	-0.889442	-0.046516	-0.253649	-0.565425	-0.779376	-0.792391	-0.772836	-0.811409	-0.871927	-0.779376	-0.935785	-0.771497	-0.284627	-0.898859	0.125293	-0.307009	-0.625294	0.008684	0.002186	0.008726	-0.005981	-0.489547	0.058593	-0.056515
std	0.070261	0.040811	0.056635	0.448734	0.502645	0.418687	0.424073	0.485942	0.414122	0.544547	0.282243	0.293867	0.363594	0.343611	0.297818	0.471808	0.259607	0.126333	0.221983	0.371608	0.371581	0.386569	0.468959	0.437268	0.371363	0.306507	0.246593	0.243635	0.230067	0.257383	0.215001	0.208837	0.221432	0.286081	0.216289	0.236226	0.232757	0.363155	0.331122	0.385379	...	0.230580	0.370612	0.670112	0.188636	0.253755	0.358631	0.345329	0.335026	0.322850	0.343454	0.293098	0.173574	0.335026	0.192684	0.318706	0.611793	0.157653	0.282665	0.326624	0.326620	0.275733	0.265434	0.287613	0.246680	0.193344	0.275733	0.138683	0.287577	0.630896	0.143135	0.250994	0.321011	0.307584	0.336787	0.448306	0.608303	0.477975	0.511807	0.297480	0.279122
min	-1.000000	-1.000000	-1.000000	-1.000000	-0.999873	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-0.999999	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-0.925897	-0.963099	-1.000000	-0.822053	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-0.753754	-1.000000	-1.000000	-1.000000	-1.000000	-0.972219	...	-1.000000	-0.999499	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-0.997500	-1.000000	-1.000000	-0.999996	-1.000000	-1.000000	-1.000000	-1.000000	-0.999996	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-0.995357	-0.999765	-0.976580	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000	-1.000000
25%	0.262975	-0.024863	-0.120993	-0.992754	-0.978129	-0.980233	-0.993591	-0.978162	-0.980251	-0.936219	-0.563561	-0.812744	0.197051	0.101829	0.389787	-0.982992	-0.999936	-0.999786	-0.999460	-0.994387	-0.982159	-0.979623	-0.573441	-0.559584	-0.505512	-0.364926	-0.082544	-0.190581	-0.023857	-0.221943	-0.135433	0.034430	-0.168841	-0.207296	-0.123514	-0.106542	-0.238877	-0.373937	-0.403511	-0.147970	...	-0.999862	-0.989044	-1.000000	-0.968254	-0.000409	-0.607485	-0.882254	-0.984123	-0.980329	-0.980453	-0.981029	-0.994189	-0.984123	-0.999725	-0.986633	-0.690397	-1.000000	-0.240882	-0.492028	-0.802207	-0.993100	-0.993508	-0.992736	-0.994355	-0.994065	-0.993100	-0.999957	-0.992268	-0.955696	-0.968254	-0.023692	-0.542602	-0.845573	-0.121527	-0.289549	-0.482273	-0.376341	-0.812065	-0.017885	-0.143414
50%	0.277193	-0.017219	-0.108676	-0.946196	-0.851897	-0.859365	-0.950709	-0.857328	-0.857143	-0.881637	-0.479677	-0.736516	0.792060	0.627737	0.778059	-0.885461	-0.998046	-0.994065	-0.985546	-0.957859	-0.896093	-0.864515	-0.073369	-0.136793	-0.148889	-0.129393	0.070073	-0.019001	0.134149	-0.040710	0.011748	0.168444	-0.022448	0.029044	0.003266	0.049432	-0.081940	-0.163728	-0.189673	0.147482	...	-0.997494	-0.950502	-0.740806	-0.904762	0.168645	-0.364784	-0.727332	-0.886439	-0.837698	-0.859053	-0.836296	-0.960136	-0.886439	-0.986266	-0.923298	-0.190223	-0.948718	-0.061597	-0.309252	-0.655594	-0.952398	-0.947140	-0.945223	-0.950796	-0.975676	-0.952398	-0.998471	-0.949768	-0.455569	-0.904762	0.134000	-0.343685	-0.711692	0.009509	0.008943	0.008735	-0.000368	-0.709417	0.182071	0.003181
75%	0.288461	-0.010783	-0.097794	-0.242813	-0.034231	-0.262415	-0.292680	-0.066701	-0.265671	-0.017129	-0.065364	-0.332014	0.844420	0.685622	0.837323	-0.107428	-0.710707	-0.816703	-0.748018	-0.393220	-0.310548	-0.316037	0.336504	0.280170	0.164123	0.132657	0.276872	0.128635	0.285318	0.172344	0.177832	0.293410	0.130862	0.230983	0.175602	0.195556	0.079229	0.070818	0.005170	0.382231	...	-0.731425	-0.371989	0.350640	-0.873016	0.364240	-0.082569	-0.449997	-0.438439	-0.451900	-0.397123	-0.540666	-0.837425	-0.438439	-0.800743	-0.484202	0.515789	-0.846154	0.148795	-0.069952	-0.428376	-0.611952	-0.642627	-0.608047	-0.688776	-0.808263	-0.611952	-0.922503	-0.603952	0.336785	-0.873016	0.289096	-0.126979	-0.503878	0.150865	0.292861	0.506187	0.359368	-0.509079	0.248353	0.107659
max	1.000000	1.000000	1.000000	1.000000	0.916238	1.000000	1.000000	0.967664	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	0.919662	1.000000	1.000000	1.000000	0.978449	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	0.814623	1.000000	0.997207	1.000000	1.000000	1.000000	1.000000	...	1.000000	1.000000	1.000000	1.000000	0.975821	1.000000	1.000000	1.000000	1.000000	1.000000	0.842119	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	0.969311	0.949350	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000	0.968254	0.946700	0.989538	0.956845	1.000000	1.000000	0.998702	0.996078	1.000000	0.478157	1.000000

8 rows × 561 columns

2.a.iv) Summarize the levels of the class attribute

Xy_train_df.groupby('targetVar').size()

targetVar
1    1226
2    1073
3     986
4    1286
5    1374
6    1407
dtype: int64

2.b) Data Visualization

# Histograms for each attribute
X_train_df.hist(layout=(dispRow,dispCol))
plt.show()

# Box and Whisker plot for each attribute
X_train_df.plot(kind='box', subplots=True, layout=(dispRow,dispCol))
plt.show()

# Correlation matrix
# fig = plt.figure(figsize=(16,12))
# ax = fig.add_subplot(111)
# correlations = X_train_df.corr(method='pearson')
# cax = ax.matshow(correlations, vmin=-1, vmax=1)
# fig.colorbar(cax)
# plt.show()

Task 3. Prepare Data

3.a) Splitting Data into Training and Test Sets

print("X_train_df.shape: {} y_train_df.shape: {}".format(X_train_df.shape, y_train_df.shape))
print("X_test_df.shape: {} y_test_df.shape: {}".format(X_test_df.shape, y_test_df.shape))

X_train_df.shape: (7352, 561) y_train_df.shape: (7352, 1)
X_test_df.shape: (2947, 561) y_test_df.shape: (2947, 1)

3.b) Feature Scaling and Data Pre-Processing

# Not applicable for this iteration of the project

3.c) Training Data Balancing

# Not applicable for this iteration of the project

3.d) Feature Selection

# Not applicable for this iteration of the project

3.e) Display the Final Datasets for Model-Building

# Finalize the training and testing datasets for the modeling activities
X_train = X_train_df.to_numpy()
y_train = y_train_df.to_numpy().ravel()
X_test = X_test_df.to_numpy()
y_test = y_test_df.to_numpy().ravel()
print("X_train.shape: {} y_train.shape: {}".format(X_train.shape, y_train.shape))
print("X_test.shape: {} y_test.shape: {}".format(X_test.shape, y_test.shape))

X_train.shape: (7352, 561) y_train.shape: (7352,)
X_test.shape: (2947, 561) y_test.shape: (2947,)

Task 4. Model and Evaluate Algorithms

4.a) Set test options and evaluation metric

# Set up Algorithms Spot-Checking Array
startTimeModule = datetime.now()
train_models = []
train_results = []
train_model_names = []
train_metrics = []
# train_models.append(('LGR', LogisticRegression(random_state=seedNum)))
# train_models.append(('CART', DecisionTreeClassifier(random_state=seedNum)))
# train_models.append(('KNN', KNeighborsClassifier(n_jobs=n_jobs)))
# train_models.append(('BGT', BaggingClassifier(random_state=seedNum, n_jobs=n_jobs)))
# train_models.append(('RNF', RandomForestClassifier(random_state=seedNum, n_jobs=n_jobs)))
# train_models.append(('EXT', ExtraTreesClassifier(random_state=seedNum, n_jobs=n_jobs)))
# train_models.append(('GBM', GradientBoostingClassifier(random_state=seedNum)))
train_models.append(('XGB', XGBClassifier(random_state=seedNum, objective='multi:softmax', num_class=6, tree_method='gpu_hist')))

# Generate model in turn
for name, model in train_models:
	if notifyStatus: email_notify("Algorithm "+name+" modeling has begun! "+datetime.now().strftime('%a %B %d, %Y %I:%M:%S %p'))
	startTimeModule = datetime.now()
	kfold = KFold(n_splits=n_folds, shuffle=True, random_state=seedNum)
	cv_results = cross_val_score(model, X_train, y_train, cv=kfold, scoring=scoring, verbose=1)
	train_results.append(cv_results)
	train_model_names.append(name)
	train_metrics.append(cv_results.mean())
	print("%s: %f (%f)" % (name, cv_results.mean(), cv_results.std()))
	print(model)
	print ('Model training time:', (datetime.now() - startTimeModule), '\n')
	if notifyStatus: email_notify("Algorithm "+name+" modeling completed! "+datetime.now().strftime('%a %B %d, %Y %I:%M:%S %p'))
print ('Average metrics ('+scoring+') from all models:',np.mean(train_metrics))
print ('Total training time for all models:',(datetime.now() - startTimeModule))

[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.

XGB: 0.987894 (0.002699)
XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
              colsample_bynode=1, colsample_bytree=1, gamma=0,
              learning_rate=0.1, max_delta_step=0, max_depth=3,
              min_child_weight=1, missing=None, n_estimators=100, n_jobs=1,
              nthread=None, num_class=6, objective='multi:softmax',
              random_state=888, reg_alpha=0, reg_lambda=1, scale_pos_weight=1,
              seed=None, silent=None, subsample=1, tree_method='gpu_hist',
              verbosity=1)
Model training time: 0:00:10.067192 

Average metrics (accuracy) from all models: 0.9878943936514102
Total training time for all models: 0:00:10.067936

[Parallel(n_jobs=1)]: Done   5 out of   5 | elapsed:   10.1s finished

4.b) Spot-checking baseline algorithms

fig = plt.figure(figsize=(16,12))
fig.suptitle('Algorithm Comparison - Spot Checking')
ax = fig.add_subplot(111)
plt.boxplot(train_results)
ax.set_xticklabels(train_model_names)
plt.show()

Task 5. Improve Accuracy

5.a) Algorithm Tuning

# Set up the comparison array
tune_results = []
tune_model_names = []

# Tuning XGBoost n_estimators, max_depth, and min_child_weight
startTimeModule = datetime.now()
if notifyStatus: email_notify("Algorithm tuning iteration #1 has begun! "+datetime.now().strftime('%a %B %d, %Y %I:%M:%S %p'))

tune_model1 = XGBClassifier(random_state=seedNum, objective='multi:softmax', num_class=6, tree_method='gpu_hist')
tune_model_names.append('XGB_1')
paramGrid1 = dict(n_estimators=range(100,1001,100), max_depth=np.array([3,6,9]), min_child_weight=np.array([1,2,3]))

kfold = KFold(n_splits=n_folds, shuffle=True, random_state=seedNum)
grid1 = GridSearchCV(estimator=tune_model1, param_grid=paramGrid1, scoring=scoring, cv=kfold, verbose=2)
grid_result1 = grid1.fit(X_train, y_train)

print("Best: %f using %s" % (grid_result1.best_score_, grid_result1.best_params_))
tune_results.append(grid_result1.cv_results_['mean_test_score'])
means = grid_result1.cv_results_['mean_test_score']
stds = grid_result1.cv_results_['std_test_score']
params = grid_result1.cv_results_['params']
for mean, stdev, param in zip(means, stds, params):
    print("%f (%f) with: %r" % (mean, stdev, param))
print ('Model training time:',(datetime.now() - startTimeModule))
if notifyStatus: email_notify("Algorithm tuning iteration #1 completed! "+datetime.now().strftime('%a %B %d, %Y %I:%M:%S %p'))

Fitting 5 folds for each of 90 candidates, totalling 450 fits
[CV] max_depth=3, min_child_weight=1, n_estimators=100 ...............

[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.

[CV]  max_depth=3, min_child_weight=1, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=1, n_estimators=100 ...............

[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    1.8s remaining:    0.0s

[CV]  max_depth=3, min_child_weight=1, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=200, total=   3.1s
[CV] max_depth=3, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=200, total=   3.1s
[CV] max_depth=3, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=200, total=   3.1s
[CV] max_depth=3, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=200, total=   3.1s
[CV] max_depth=3, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=200, total=   3.1s
[CV] max_depth=3, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=300, total=   4.3s
[CV] max_depth=3, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=300, total=   4.3s
[CV] max_depth=3, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=300, total=   4.3s
[CV] max_depth=3, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=300, total=   4.3s
[CV] max_depth=3, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=300, total=   4.3s
[CV] max_depth=3, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=400, total=   5.4s
[CV] max_depth=3, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=400, total=   5.4s
[CV] max_depth=3, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=400, total=   5.4s
[CV] max_depth=3, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=400, total=   5.4s
[CV] max_depth=3, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=400, total=   5.4s
[CV] max_depth=3, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=500, total=   6.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=500, total=   6.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=500, total=   6.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=500, total=   6.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=500, total=   6.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=600, total=   7.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=600, total=   7.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=600, total=   7.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=600, total=   7.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=600, total=   7.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=700, total=   8.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=700, total=   8.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=700, total=   8.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=700, total=   8.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=700, total=   8.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=800, total=   9.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=800, total=   9.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=800, total=   9.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=800, total=   9.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=800, total=   9.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=900, total=  10.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=900, total=  10.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=900, total=  10.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=900, total=  10.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=1, n_estimators=900, total=  10.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=1, n_estimators=1000, total=  11.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=1, n_estimators=1000, total=  11.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=1, n_estimators=1000, total=  11.4s
[CV] max_depth=3, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=1, n_estimators=1000, total=  11.5s
[CV] max_depth=3, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=1, n_estimators=1000, total=  11.4s
[CV] max_depth=3, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=300, total=   4.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=300, total=   4.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=300, total=   4.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=300, total=   4.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=300, total=   4.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=500, total=   6.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=500, total=   6.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=500, total=   6.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=500, total=   6.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=500, total=   6.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=600, total=   7.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=600, total=   7.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=600, total=   7.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=600, total=   7.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=600, total=   7.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=700, total=   8.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=700, total=   8.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=700, total=   8.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=700, total=   8.3s
[CV] max_depth=3, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=700, total=   8.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=800, total=   9.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=800, total=   9.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=800, total=   9.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=800, total=   9.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=800, total=   9.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=900, total=  10.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=900, total=  10.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=900, total=  10.1s
[CV] max_depth=3, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=900, total=  10.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=2, n_estimators=900, total=  10.2s
[CV] max_depth=3, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=2, n_estimators=1000, total=  11.1s
[CV] max_depth=3, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=2, n_estimators=1000, total=  11.1s
[CV] max_depth=3, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=2, n_estimators=1000, total=  11.1s
[CV] max_depth=3, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=2, n_estimators=1000, total=  11.1s
[CV] max_depth=3, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=2, n_estimators=1000, total=  11.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=100, total=   1.8s
[CV] max_depth=3, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=200, total=   3.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=300, total=   4.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=300, total=   4.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=300, total=   4.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=300, total=   4.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=300, total=   4.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=400, total=   5.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=400, total=   5.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=500, total=   6.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=500, total=   6.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=500, total=   6.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=500, total=   6.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=500, total=   6.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=600, total=   7.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=600, total=   7.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=600, total=   7.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=600, total=   7.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=600, total=   7.2s
[CV] max_depth=3, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=700, total=   8.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=700, total=   8.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=700, total=   8.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=700, total=   8.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=700, total=   8.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=800, total=   9.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=800, total=   9.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=800, total=   9.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=800, total=   9.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=800, total=   9.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=900, total=  10.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=900, total=  10.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=900, total=  10.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=900, total=  10.1s
[CV] max_depth=3, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=3, min_child_weight=3, n_estimators=900, total=  10.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=3, n_estimators=1000, total=  11.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=3, n_estimators=1000, total=  11.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=3, n_estimators=1000, total=  11.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=3, n_estimators=1000, total=  11.0s
[CV] max_depth=3, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=3, min_child_weight=3, n_estimators=1000, total=  11.0s
[CV] max_depth=6, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=100, total=   2.7s
[CV] max_depth=6, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=100, total=   2.8s
[CV] max_depth=6, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=100, total=   2.7s
[CV] max_depth=6, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=100, total=   2.8s
[CV] max_depth=6, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=100, total=   2.8s
[CV] max_depth=6, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=200, total=   4.1s
[CV] max_depth=6, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=200, total=   4.1s
[CV] max_depth=6, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=200, total=   4.1s
[CV] max_depth=6, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=200, total=   4.1s
[CV] max_depth=6, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=200, total=   4.1s
[CV] max_depth=6, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=300, total=   5.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=300, total=   5.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=300, total=   5.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=300, total=   5.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=300, total=   5.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=400, total=   6.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=400, total=   6.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=400, total=   6.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=400, total=   6.4s
[CV] max_depth=6, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=400, total=   6.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=500, total=   7.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=500, total=   7.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=500, total=   7.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=500, total=   7.4s
[CV] max_depth=6, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=500, total=   7.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=600, total=   8.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=600, total=   8.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=600, total=   8.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=600, total=   8.4s
[CV] max_depth=6, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=600, total=   8.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=700, total=   9.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=700, total=   9.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=700, total=   9.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=700, total=   9.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=700, total=   9.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=800, total=  10.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=800, total=  10.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=800, total=  10.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=800, total=  10.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=800, total=  10.3s
[CV] max_depth=6, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=900, total=  11.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=900, total=  11.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=900, total=  11.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=900, total=  11.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=1, n_estimators=900, total=  11.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=1, n_estimators=1000, total=  12.1s
[CV] max_depth=6, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=1, n_estimators=1000, total=  12.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=1, n_estimators=1000, total=  12.1s
[CV] max_depth=6, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=1, n_estimators=1000, total=  12.2s
[CV] max_depth=6, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=1, n_estimators=1000, total=  12.2s
[CV] max_depth=6, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=100, total=   2.6s
[CV] max_depth=6, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=100, total=   2.6s
[CV] max_depth=6, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=100, total=   2.5s
[CV] max_depth=6, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=100, total=   2.6s
[CV] max_depth=6, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=100, total=   2.6s
[CV] max_depth=6, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=200, total=   3.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=200, total=   3.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=200, total=   3.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=200, total=   3.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=200, total=   3.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=300, total=   4.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=300, total=   4.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=300, total=   4.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=300, total=   4.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=300, total=   4.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=400, total=   5.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=400, total=   5.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=400, total=   5.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=400, total=   5.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=400, total=   5.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=500, total=   6.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=500, total=   6.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=500, total=   6.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=500, total=   6.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=500, total=   6.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=600, total=   7.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=600, total=   7.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=600, total=   7.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=600, total=   7.9s
[CV] max_depth=6, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=600, total=   7.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=700, total=   8.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=700, total=   8.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=700, total=   8.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=700, total=   8.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=700, total=   8.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=800, total=   9.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=800, total=   9.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=800, total=   9.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=800, total=   9.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=800, total=   9.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=900, total=  10.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=900, total=  10.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=900, total=  10.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=900, total=  10.8s
[CV] max_depth=6, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=2, n_estimators=900, total=  10.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=2, n_estimators=1000, total=  11.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=2, n_estimators=1000, total=  11.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=2, n_estimators=1000, total=  11.6s
[CV] max_depth=6, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=2, n_estimators=1000, total=  11.7s
[CV] max_depth=6, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=2, n_estimators=1000, total=  11.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=100, total=   2.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=100, total=   2.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=100, total=   2.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=100, total=   2.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=100, total=   2.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=200, total=   3.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=200, total=   3.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=200, total=   3.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=200, total=   3.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=200, total=   3.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=300, total=   4.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=300, total=   4.8s
[CV] max_depth=6, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=300, total=   4.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=300, total=   4.8s
[CV] max_depth=6, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=300, total=   4.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=400, total=   5.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=400, total=   5.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=400, total=   5.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=400, total=   5.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=400, total=   5.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=500, total=   6.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=500, total=   6.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=500, total=   6.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=500, total=   6.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=500, total=   6.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=600, total=   7.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=600, total=   7.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=600, total=   7.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=600, total=   7.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=600, total=   7.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=700, total=   8.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=700, total=   8.7s
[CV] max_depth=6, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=700, total=   8.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=700, total=   8.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=700, total=   8.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=800, total=   9.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=800, total=   9.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=800, total=   9.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=800, total=   9.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=800, total=   9.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=900, total=  10.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=900, total=  10.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=900, total=  10.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=900, total=  10.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=6, min_child_weight=3, n_estimators=900, total=  10.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=3, n_estimators=1000, total=  11.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=3, n_estimators=1000, total=  11.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=3, n_estimators=1000, total=  11.5s
[CV] max_depth=6, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=3, n_estimators=1000, total=  11.6s
[CV] max_depth=6, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=6, min_child_weight=3, n_estimators=1000, total=  11.5s
[CV] max_depth=9, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=100, total=   3.3s
[CV] max_depth=9, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=100, total=   3.3s
[CV] max_depth=9, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=100, total=   3.3s
[CV] max_depth=9, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=100, total=   3.3s
[CV] max_depth=9, min_child_weight=1, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=100, total=   3.3s
[CV] max_depth=9, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=200, total=   4.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=200, total=   4.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=200, total=   4.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=200, total=   4.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=200, total=   4.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=300, total=   5.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=300, total=   5.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=300, total=   5.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=300, total=   5.8s
[CV] max_depth=9, min_child_weight=1, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=300, total=   5.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=400, total=   6.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=400, total=   6.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=400, total=   6.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=400, total=   6.8s
[CV] max_depth=9, min_child_weight=1, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=400, total=   6.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=500, total=   7.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=500, total=   7.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=500, total=   7.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=500, total=   7.8s
[CV] max_depth=9, min_child_weight=1, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=500, total=   7.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=600, total=   8.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=600, total=   8.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=600, total=   8.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=600, total=   8.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=600, total=   8.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=700, total=   9.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=700, total=   9.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=700, total=   9.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=700, total=   9.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=700, total=   9.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=800, total=  10.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=800, total=  10.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=800, total=  10.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=800, total=  10.7s
[CV] max_depth=9, min_child_weight=1, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=800, total=  10.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=900, total=  11.5s
[CV] max_depth=9, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=900, total=  11.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=900, total=  11.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=900, total=  11.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=1, n_estimators=900, total=  11.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=1, n_estimators=1000, total=  12.5s
[CV] max_depth=9, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=1, n_estimators=1000, total=  12.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=1, n_estimators=1000, total=  12.5s
[CV] max_depth=9, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=1, n_estimators=1000, total=  12.6s
[CV] max_depth=9, min_child_weight=1, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=1, n_estimators=1000, total=  12.5s
[CV] max_depth=9, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=100, total=   3.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=100, total=   3.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=100, total=   3.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=100, total=   3.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=100, total=   3.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=200, total=   4.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=200, total=   4.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=200, total=   4.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=200, total=   4.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=200, total=   4.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=300, total=   5.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=300, total=   5.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=300, total=   5.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=300, total=   5.3s
[CV] max_depth=9, min_child_weight=2, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=300, total=   5.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=400, total=   6.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=400, total=   6.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=400, total=   6.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=400, total=   6.3s
[CV] max_depth=9, min_child_weight=2, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=400, total=   6.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=500, total=   7.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=500, total=   7.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=500, total=   7.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=500, total=   7.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=500, total=   7.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=600, total=   8.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=600, total=   8.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=600, total=   8.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=600, total=   8.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=600, total=   8.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=700, total=   9.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=700, total=   9.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=700, total=   9.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=700, total=   9.2s
[CV] max_depth=9, min_child_weight=2, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=700, total=   9.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=800, total=  10.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=800, total=  10.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=800, total=  10.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=800, total=  10.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=800, total=  10.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=900, total=  11.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=900, total=  11.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=900, total=  11.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=900, total=  11.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=2, n_estimators=900, total=  11.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=2, n_estimators=1000, total=  12.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=2, n_estimators=1000, total=  12.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=2, n_estimators=1000, total=  12.0s
[CV] max_depth=9, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=2, n_estimators=1000, total=  12.1s
[CV] max_depth=9, min_child_weight=2, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=2, n_estimators=1000, total=  12.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=100, total=   2.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=100, total=   2.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=100, total=   2.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=100, total=   2.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=100 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=100, total=   2.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=200, total=   4.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=200, total=   4.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=200, total=   4.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=200, total=   4.1s
[CV] max_depth=9, min_child_weight=3, n_estimators=200 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=200, total=   4.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=300, total=   5.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=300, total=   5.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=300, total=   5.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=300, total=   5.1s
[CV] max_depth=9, min_child_weight=3, n_estimators=300 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=300, total=   5.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=400, total=   6.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=400, total=   6.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=400, total=   6.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=400, total=   6.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=400 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=400, total=   6.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=500, total=   6.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=500, total=   6.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=500, total=   6.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=500, total=   7.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=500 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=500, total=   7.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=600, total=   7.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=600, total=   7.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=600, total=   7.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=600, total=   8.0s
[CV] max_depth=9, min_child_weight=3, n_estimators=600 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=600, total=   7.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=700, total=   8.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=700, total=   8.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=700, total=   8.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=700, total=   8.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=700 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=700, total=   8.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=800, total=   9.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=800, total=   9.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=800, total=   9.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=800, total=   9.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=800 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=800, total=   9.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=900, total=  10.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=900, total=  10.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=900, total=  10.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=900, total=  10.9s
[CV] max_depth=9, min_child_weight=3, n_estimators=900 ...............
[CV]  max_depth=9, min_child_weight=3, n_estimators=900, total=  10.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=3, n_estimators=1000, total=  11.7s
[CV] max_depth=9, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=3, n_estimators=1000, total=  11.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=3, n_estimators=1000, total=  11.7s
[CV] max_depth=9, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=3, n_estimators=1000, total=  11.8s
[CV] max_depth=9, min_child_weight=3, n_estimators=1000 ..............
[CV]  max_depth=9, min_child_weight=3, n_estimators=1000, total=  11.8s

[Parallel(n_jobs=1)]: Done 450 out of 450 | elapsed: 54.5min finished

Best: 0.994559 using {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 600}
0.987894 (0.002699) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 100}
0.992927 (0.001697) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 200}
0.993879 (0.000746) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 300}
0.994015 (0.000902) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 400}
0.994423 (0.000998) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 500}
0.994559 (0.000744) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 600}
0.994423 (0.000998) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 700}
0.994423 (0.000998) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 800}
0.994423 (0.000998) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 900}
0.994559 (0.000744) with: {'max_depth': 3, 'min_child_weight': 1, 'n_estimators': 1000}
0.988846 (0.003293) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 100}
0.992927 (0.001999) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 200}
0.993471 (0.001645) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 300}
0.993471 (0.001753) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 400}
0.993743 (0.001632) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 500}
0.993743 (0.001632) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 600}
0.993743 (0.001632) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 700}
0.994151 (0.001400) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 800}
0.994151 (0.001400) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 900}
0.994287 (0.001400) with: {'max_depth': 3, 'min_child_weight': 2, 'n_estimators': 1000}
0.988031 (0.002903) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 100}
0.992111 (0.002043) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 200}
0.993199 (0.001773) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 300}
0.993471 (0.001642) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 400}
0.993607 (0.001951) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 500}
0.993743 (0.001844) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 600}
0.994015 (0.001687) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 700}
0.993743 (0.001844) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 800}
0.993743 (0.001844) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 900}
0.993743 (0.001844) with: {'max_depth': 3, 'min_child_weight': 3, 'n_estimators': 1000}
0.989391 (0.001525) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 100}
0.991431 (0.001185) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 200}
0.992247 (0.001104) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 300}
0.992383 (0.001168) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 400}
0.992383 (0.000901) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 500}
0.992383 (0.000901) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 600}
0.992519 (0.000961) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 700}
0.992519 (0.000961) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 800}
0.992519 (0.000961) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 900}
0.992519 (0.000961) with: {'max_depth': 6, 'min_child_weight': 1, 'n_estimators': 1000}
0.990071 (0.001464) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 100}
0.991703 (0.001688) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 200}
0.992111 (0.001527) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 300}
0.992383 (0.001688) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 400}
0.992383 (0.001688) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 500}
0.992383 (0.001688) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 600}
0.992655 (0.001688) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 700}
0.992791 (0.001527) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 800}
0.992791 (0.001527) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 900}
0.992791 (0.001261) with: {'max_depth': 6, 'min_child_weight': 2, 'n_estimators': 1000}
0.989663 (0.001316) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 100}
0.991567 (0.001752) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 200}
0.991703 (0.001451) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 300}
0.992111 (0.001104) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 400}
0.992111 (0.001104) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 500}
0.992111 (0.001104) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 600}
0.992111 (0.001104) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 700}
0.992111 (0.001104) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 800}
0.992383 (0.000792) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 900}
0.992383 (0.000792) with: {'max_depth': 6, 'min_child_weight': 3, 'n_estimators': 1000}
0.987351 (0.001698) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 100}
0.988983 (0.002485) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 200}
0.989663 (0.002630) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 300}
0.989799 (0.002471) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 400}
0.989799 (0.002471) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 500}
0.989663 (0.002292) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 600}
0.989799 (0.002150) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 700}
0.989799 (0.002150) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 800}
0.989935 (0.002292) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 900}
0.989935 (0.002292) with: {'max_depth': 9, 'min_child_weight': 1, 'n_estimators': 1000}
0.987623 (0.002166) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 100}
0.989935 (0.002485) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 200}
0.990615 (0.002167) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 300}
0.990615 (0.002167) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 400}
0.990479 (0.002150) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 500}
0.990479 (0.002150) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 600}
0.990479 (0.002316) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 700}
0.990615 (0.002371) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 800}
0.990615 (0.002371) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 900}
0.990751 (0.002493) with: {'max_depth': 9, 'min_child_weight': 2, 'n_estimators': 1000}
0.988166 (0.002134) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 100}
0.990207 (0.001855) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 200}
0.990615 (0.002292) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 300}
0.990615 (0.002292) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 400}
0.991023 (0.001989) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 500}
0.991159 (0.002017) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 600}
0.991295 (0.001990) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 700}
0.991431 (0.001999) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 800}
0.991295 (0.001990) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 900}
0.991703 (0.001742) with: {'max_depth': 9, 'min_child_weight': 3, 'n_estimators': 1000}
Model training time: 0:54:38.900996

5.b) Compare Algorithms After Tuning

fig = plt.figure(figsize=(16,12))
fig.suptitle('Algorithm Comparison - Post Tuning')
ax = fig.add_subplot(111)
plt.boxplot(tune_results)
ax.set_xticklabels(tune_model_names)
plt.show()

Task 6. Finalize Model

6.a) Measure predictions from the test dataset

test_model = XGBClassifier(n_estimators=600, max_depth=3, min_child_weight=1, random_state=seedNum, objective='multi:softmax', num_class=6, tree_method='gpu_hist')
test_model.fit(X_train, y_train)
predictions = test_model.predict(X_test)
print('Accuracy Score:', accuracy_score(y_test, predictions))
print(confusion_matrix(y_test, predictions))
print(classification_report(y_test, predictions))
print(test_model)

Accuracy Score: 0.9494401085850017
[[491   4   1   0   0   0]
 [ 28 437   6   0   0   0]
 [  7  17 396   0   0   0]
 [  0   2   0 429  60   0]
 [  0   0   0  24 508   0]
 [  0   0   0   0   0 537]]
              precision    recall  f1-score   support

           1       0.93      0.99      0.96       496
           2       0.95      0.93      0.94       471
           3       0.98      0.94      0.96       420
           4       0.95      0.87      0.91       491
           5       0.89      0.95      0.92       532
           6       1.00      1.00      1.00       537

    accuracy                           0.95      2947
   macro avg       0.95      0.95      0.95      2947
weighted avg       0.95      0.95      0.95      2947

XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
              colsample_bynode=1, colsample_bytree=1, gamma=0,
              learning_rate=0.1, max_delta_step=0, max_depth=3,
              min_child_weight=1, missing=None, n_estimators=600, n_jobs=1,
              nthread=None, num_class=6, objective='multi:softprob',
              random_state=888, reg_alpha=0, reg_lambda=1, scale_pos_weight=1,
              seed=None, silent=None, subsample=1, tree_method='gpu_hist',
              verbosity=1)

6.b) Create a standalone model using all available data

# Combining the training and testing datasets to form the complete dataset that will be used for training the final model
# X_complete = np.vstack((X_train, X_test))
# y_complete = np.concatenate((y_train, y_test))
# print("X_complete.shape: {} y_complete.shape: {}".format(X_complete.shape, y_complete.shape))
# final_model = test_model1.fit(X_complete, y_complete)
# print(final_model)

6.c) Save the final model for later use

# modelName = 'FinalModel_BinaryClass.sav'
# dump(final_model, modelName)

print ('Total time for the script:',(datetime.now() - startTimeScript))

Total time for the script: 0:57:08.492687